Image quality evaluation

ABSTRACT

A method of evaluating image quality includes capturing a first digital image of a target object using a first digital camera of a smart device. A second digital image of a user of the smart device is captured using a second digital camera of the smart device. The second digital image includes an image of the user&#39;s facial expression. A quality index is generated for the first digital image by analyzing one or more features of the second digital image. Analyzing the second digital image includes determining the user&#39;s sentiment. The quality index is then associated with the first digital image.

BACKGROUND

The present invention generally relates to photography, and moreparticularly to image quality evaluation.

Smart devices, such as smart phones and tablets may be equipped with aplurality of cameras. For example, users of smart devices may capturedigital images of themselves, other individuals or objects in asurrounding environment. Thus, users of smart devices often capturerelatively large number of images with varying degrees of subjectivesuccess. For example, captured images may be classified in user-createdfolders. Image content can be analyzed to classify by theme (e.g. familyphotos, scenery, etc.), or the images may be time and geography tagged.In addition, some information provided by viewers can be added, such asthe “like” tags that have become common practice on social media.

The relatively large numbers of captured digital images may be stored onthe smart devices or in an image database that may be accessible at alater time. Digital images that are captured and stored in suchdatabases may include metadata. The metadata may include additionalinformation about the captured digital images, such as a time or placethat the digital image was captured. Metadata may include words orphrases associated with the images, or may be used to capture desireddata that a user may wish to associate with a particular digital image.Thus, the metadata may be used as a way to identify one or more desireddigital images for later search, retrieval and display to a user. Forexample, previously captured images may be captured by a user based on aparticular search query, and thus a particular subset of images matchingthe search query may be displayed to a user.

SUMMARY

According to an exemplary embodiment of the present invention, acomputer-implemented method of evaluating photographer satisfactionincludes capturing a first digital image of a target object using afirst digital camera of a smart device. A second digital image of a userof the smart device is captured using a second digital camera of thesmart device. The second digital image includes an image of the user'sfacial expression. A satisfaction index is generated for the firstdigital image by analyzing the user's sentiment related to the firstdigital image. Analyzing the user's sentiment related to the firstdigital image includes evaluating the user's facial expression. Thesatisfaction index is then associated with the first digital image.

According to an exemplary embodiment of the present invention, a methodof evaluating photographer satisfaction includes capturing a firstdigital image of a target object using a first digital camera of a smartdevice. A second digital image of a user of the smart device is capturedusing a second digital camera of the smart device. The second digitalimage includes an image of the user's facial expression. The methodincludes determining if there was an interaction between the user andthe target object. A satisfaction index for the first digital image isgenerated by analyzing the user's sentiment related to the first digitalimage. Analyzing the user's sentiment related to the first digital imageincludes evaluating the user's facial expression. The satisfaction indexis then associated with the first digital image.

According to an exemplary embodiment of the present invention, acomputer program product includes a computer readable storage mediumhaving program instructions embodied therewith. The program instructionsare executable by a processor to cause the processor to capture a firstdigital image of a target object using a first digital camera of a smartdevice. The program instructions are executable by a processor to causethe processor to capture a second digital image of a user of the smartdevice using a second digital camera of the smart device. The seconddigital image includes an image of the user's facial expression. Theprogram instructions are executable by a processor to cause theprocessor to generate a satisfaction index for the first digital imageby analyzing the user's sentiment related to the first digital image.Analyzing the user's sentiment related to the first digital imageincludes evaluating the user's facial expression. The programinstructions are executable by a processor to cause the processor toassociate the satisfaction index with the first digital image.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features of the present invention will become moreapparent by describing in detail exemplary embodiments thereof, withreference to the accompanying drawings, in which:

FIG. 1A illustrates a user of a smart device capturing an imageaccording to an exemplary embodiment of the present invention.

FIG. 1B illustrates a back side of a smart device including a firstcamera according to an exemplary embodiment of the present invention.

FIG. 1C illustrates a front side of a smart device including a secondcamera according to an exemplary embodiment of the present invention.

FIG. 2 illustrates a method of evaluating photographer satisfactionaccording to an exemplary embodiment of the present invention.

FIG. 3 illustrates a method of evaluating photographer satisfactionaccording to an exemplary embodiment of the present invention.

FIG. 4 illustrates a method of evaluating photographer satisfactionaccording to an exemplary embodiment of the present invention.

FIG. 5 illustrates an example of a computer system capable ofimplementing the methods according to exemplary embodiments of thepresent invention.

DETAILED DESCRIPTION

It will be understood that the terms “first,” “second,” “third,” etc.are used herein to distinguish one element from another, and theelements are not limited by these terms. Thus, a “first” element in anexemplary embodiment may be described as a “second” element in anotherexemplary embodiment.

Exemplary embodiments of the present invention will be described morefully hereinafter with reference to the accompanying drawings. Likereference numerals may refer to like elements throughout thespecification and drawings.

The terms “photographer” and “user” may be used interchangeably herein.The terms “photograph,” “photo,” “image,” and “digital image” may beused interchangeably herein.

FIG. 1A illustrates a user of a smart device capturing an imageaccording to an exemplary embodiment of the present invention. FIG. 1Billustrates a back side of a smart device including a first cameraaccording to an exemplary embodiment of the present invention. FIG. 1Cillustrates a front side of a smart device including a second cameraaccording to an exemplary embodiment of the present invention. FIG. 2illustrates a method of evaluating photographer satisfaction accordingto an exemplary embodiment of the present invention. FIG. 3 illustratesa method of evaluating photographer satisfaction according to anexemplary embodiment of the present invention. FIG. 4 illustrates amethod of evaluating photographer satisfaction according to an exemplaryembodiment of the present invention.

Referring to FIGS. 1A, 1B, 1C, 2, 3 and 4, a computer-implemented methodof evaluating photographer satisfaction according to an exemplaryembodiment of the present invention includes capturing a first digitalimage of a target object using a first digital camera of a smart device201. A second digital image of a user of the smart device is capturedusing a second digital camera of the smart device 202. The seconddigital image includes an image of the user's facial expression. Asatisfaction index is generated for the first digital image by analyzingthe user's sentiment related to the first digital image 203. Analyzingthe user's sentiment related to the first digital image includesevaluating the user's facial expression. The satisfaction index is thenassociated with the first digital image 204.

A first digital image 301 of a target object 302 may be captured using afirst digital camera 101 of a smart device 100. A second digital image403 of a user 110 may be captured using a second digital camera 102 ofthe smart device 100. The term “smart device,” as used herein, may referto a Smartphone, Tablet, Smart Glasses, Smart Watch, or any desiredportable technology including a digital camera and carried on or about auser.

The smart device 100 may include a back side facing away from the user110 (e.g., facing toward a target object—see, e.g., FIG. 1B) and a frontside facing toward the user 110 (see, e.g., FIG. 1C). The first digitalcamera 101 may be positioned on the back side of the smart device 100facing away from the user 110 and the second digital camera 102 may bepositioned on the front side of the smart device 100 facing the user110. The first digital camera 101 may be used to capture one or morefirst digital images 301 of a target object (e.g., target object 302),while the second digital camera may be used to capture one or moresecond digital images 403 of the user 110 taking the first digital image301. Thus, as discussed below in more detail, the second digital image403 may be used to assess the user's 110 satisfaction with the firstdigital image 301

According to an exemplary embodiment of the present invention, a targetobject (e.g., target object 302) may include another person or an objectin an environment around the user 110.

According to an exemplary embodiment of the present invention, themethod of evaluating photographer satisfaction may include capturing anaudio recording of the user 110. The audio recording may be capturedusing of a second microphone 104 positioned on the front side of thesmart device 100 facing the user 110. The smart device may also includea first microphone 103 positioned on the back side of the smart device100 facing away from the user 110. The first microphone 103 may be usedto capture an audio recording of the environment (e.g., emitted frompeople or objects) around the smart device 100. Capturing the audiorecording of the user 110 may include recording the user's voice. Theuser's sentiment related to the first digital image 301 may be furtheranalyzed using the audio recording of the user 110. Generation of asatisfaction index and analysis of a user's sentiment according to anexemplary embodiment of the present invention will be described below inmore detail.

According to an exemplary embodiment of the present invention, the firstdigital image 301 and the second digital image 403 may be capturedsubstantially simultaneously. For example, the second digital image 403of the user 110 may be captured while the first digital image 301 isbeing captured.

According to an exemplary embodiment of the present invention, theuser's facial expression may be captured as a video recorded for a firstpredetermined time period before capturing the first digital image 301and for a second predetermined time period after capturing the firstdigital image 301. As an example, at least one of the first and seconddigital cameras 101 and 102 may be a video camera, or a camera equippedwith both still image and video capture capabilities.

According to an exemplary embodiment of the present invention, thedigital images captured by at least one of the first and second digitalcameras 101 and 102 may be stored locally on a hard drive of the smartdevice 100. Alternatively, the digital images captured by at least oneof the first and second digital cameras 101 and 102 may be storedremotely on a server (e.g., a cloud based server), which may beaccessible through a network interface or a network antenna of the smartdevice 100. For example, the server may be accessible by WiFi, celltower network connectivity or by Bluetooth connectivity of the smartdevice 100.

According to an exemplary embodiment of the present invention, thecaptured facial expressions and/or voice of the user 110 may generate aseparate output file that may be associated with the first digital image301.

FIG. 3 illustrates a method of evaluating photographer satisfactionaccording to an exemplary embodiment of the present invention.

Referring again to FIG. 3, according to an exemplary embodiment of thepresent invention, a method of evaluating photographer satisfactionincludes capturing a first digital image 301 of a target object 302using a first digital camera 101 of a smart device 100. A second digitalimage 403 of a user 110 of the smart device 100 is captured using asecond digital camera 102 of the smart device 100. The second digitalimage 403 includes an image of the user's facial expression. The methodincludes determining if there was an interaction between the user 110and the target object 302. A satisfaction index for the first digitalimage is generated by analyzing the user's sentiment related to thefirst digital image 301. Analyzing the user's sentiment related to thefirst digital image 301 includes evaluating the user's facialexpression. The satisfaction index is then associated with the firstdigital image 301. Generation of a satisfaction index and analysis of auser's sentiment according to an exemplary embodiment of the presentinvention will be described below in more detail.

According to an exemplary embodiment of the present invention,determining if there was an interaction between the user 110 and atarget object may include facial and voice expressions of the user 110before, during and after capturing the first digital image 301 of thetarget object. For example, the user 110 may make silly faces at a childto encourage the child to smile before capturing the first digital image301.

According to an exemplary embodiment of the present invention, themethod of evaluating photographer satisfaction may include receiving aquery for identifying at least one first digital image 301 of aplurality of first digital images. The query may include a request toretrieve an image based on a corresponding satisfaction index of the atleast one first digital image 301 of the plurality of first digitalimages. The method of evaluating photographer satisfaction may includeidentifying at least one first digital image 301 of the plurality offirst digital images based on the corresponding satisfaction indexassociated with the at least one first digital image 301 of theplurality of first digital images. For example, as described below inmore detail, the first digital image may be stored in a photo library320.

A photographer (e.g., user 110) may take at least one photograph of atarget object 302 and may then observe the resulting photograph.Observing the photograph may be done immediately following capturing thephotograph (e.g., within a few seconds). A system implementing themethods according to an exemplary embodiment of the present inventionmay use the front facing camera of a smart device used to capture theimage to record a facial expression and/or the voice of the user whilethe photograph is observed by the user. The user may express a relativedegree of satisfaction with the captured image, such as by making apositive (e.g., smiling) or negative (e.g., frowning) facial expressionwhile viewing the image. The system may analyze the facial expressionand/or voice of the user to generate a satisfaction index. Thesatisfaction index may be associated with the corresponding digitalimage (e.g., as metadata). Thus, the first digital image 301 may besearchable and may be identified based on a satisfaction index scoreapplied to the first image 301.

As an example, user satisfaction may be stated explicitly, such as bymaking an explicit declarative statement such as “this photo is great”or “I do not like this photo.” This explicit statement may be associatedwith the corresponding digital image (e.g., as metadata).

As an example, the satisfaction index may include a scale from 0 to 100,in which a score of “0” indicates the user 110 is not satisfied with thefirst image 301 at all, and a score of “100” indicates completesatisfaction with an image. Thus, the satisfaction index score may bedetermined based on a spectrum of satisfaction from 0-100. As anexample, a completely satisfied expression may be a smile displayed bythe user 110, while an expression indicating the user 110 is notsatisfied at all may be expressed as a frown. Additionally, a user 110may make an overt statement indicating a level of satisfaction (e.g., anutterance of “I am completely satisfied with this image”), an overtstatement of dissatisfaction (e.g., “I am completely dissatisfied withthis image”), or a more moderate statement of satisfaction (e.g., “I amequally satisfied with some aspects of this image and dissatisfied withsome other aspects of this image”), which may be recorded by the secondmicrophone 104 of the smart device 100. As discussed herein, the facialexpression and/or the voice recording of the user 110 may generate aseparate output file, and/or may be stored as metadata associated withthe first digital image 301. Each image including the metadata, and/orthe separate output file may be stored in the photo library 320, asdescribed herein.

The user 110 may formulate a query based on a satisfaction index scoreof a particular first digital image 301. For example, the query mayinclude a request to retrieved images having a particular satisfactionindex score (e.g., a score of 90 or greater). However, exemplaryembodiments of the present invention are not limited to a particularsearch query, and any query may be formulated by a user, as desired.Thus, an image having a satisfaction index score matching the query maybe identified (e.g., from the photo library 320).

According to an exemplary method of the present invention, thecomputer-implemented method may include receiving a query foridentifying at least one first digital image 301 of a plurality of firstdigital images. The query may include a request to retrieve an imagebased on a corresponding satisfaction index of the at least one firstdigital image 301 of the plurality of first digital images. The querymay include a request to retrieve the image based on an identifiedinteraction between the user 110 and the target object. Thecomputer-implemented method may include identifying at least one firstdigital image 301 of the plurality of first digital images based on thecorresponding satisfaction index associated with the at least one firstdigital image 301 of the plurality of first digital images, and based onthe identified interaction between the user and the target object.

FIG. 4 illustrates a method of evaluating photographer satisfactionaccording to an exemplary embodiment of the present invention. Thefeatures described above with reference to FIGS. 1A, 1B, 1C, 2 and 3 aresimilarly available in the exemplary embodiment described below withreference to FIG. 4, and thus duplicative descriptions may be omittedbelow.

Referring again to FIG. 4, a method of evaluating photographersatisfaction according to an exemplary embodiment of the presentinvention may include determining if there was an interaction betweenthe user 110 and the target object. The target object may be anotherperson in the environment around the user. Thus, a first digital image401 may be captured of a second person, and the user 110 may interactwith the second person prior to capturing the first digital image 401.For example, the second person may be a child, and the user 110 mayinteract with the child by making a silly face to encourage the child tosmile, prior to capturing the first digital image 401 of the child. Athird digital image 403 (which may be a video recording, as describedabove) of the user 110 may be captured and used to determine if the user110 interacted with the target object (e.g., child) prior to capturingthe first digital image 110 of the child. A second digital image 402 maybe captured of the user 110 during and/or after the capturing of thefirst digital image 401, as described above in more detail.

According to an exemplary embodiment of the present invention, aphotographer interaction with a target object may be captured as thethird digital image 403. The photographer may take a photograph of thetarget object (e.g., the first digital image 401) after interacting withthe target object. After capturing the first digital image 401 of thetarget object, the photographer may look at the resulting photograph andmake facial expression and/or voice a response to the first digitalimage 401. The photographer's facial expression reaction may be capturedas the second digital image 402. The photographer's voice mayadditionally be captured, which may include a statement of satisfaction.The statement of satisfaction may be an explicit statement ofsatisfaction or an implicit statement of satisfaction.

According to an exemplary embodiment of the present invention, each ofthe first, second and third digital images 401, 402 and 403 may bestored in and retrieved from a photo library 420. Additionally, whenretrieved, each of the first, second and third digital images 401, 402and 403 may be displayed in the sequence in which they were captured.Thus, a user 110 may be submit a query including a request to retrievephotographs in which the user 110 interacted with the target object, andin which the user displayed a particular satisfaction index score.

A system according to an exemplary embodiment of the present inventionmay use the front camera (e.g., the camera 102) and microphone (e.g.,the second microphone 104) of the smart device (e.g., smart device 110)to record a facial expression and voice of the photographer. The backcamera (e.g., the camera 101) of the smart device may record a photo(e.g., the first digital image 401) taken by the photographer. Thesystem may use the front camera and the microphone of the smart deviceto record the facial expression and/or voice of the photographer. Thesystem may analyze the facial expression and voice to match capturedimages and voice across pre-photo, photo and post-photo moments. Thus,the system may generate an assessment of the interactions between thephotographer and the target object, and a reaction of the photographerto the captured photograph to determine a satisfaction index score forthe first digital image 401.

Generation of a satisfaction index, and analysis of user sentiment(e.g., sentiment analysis) according to an exemplary embodiment of thepresent invention, will be described below in more detail.

According to an exemplary embodiment of the present invention, thesatisfaction index score for a captured digital image may includeevaluating the captured image using sentiment analysis.

Sentiment analysis, which may also be referred to as opinion mining oremotion artificial intelligence (emotional AI), refers to the use ofnatural language processing, text analysis, computational linguistics,and biometrics to systematically identify, extract, quantify, and studyaffective states and subjective information. Sentiment analysis may beapplied to analyze a captured image (e.g., a captured digital image)and/or captured voice recording.

Sentiment analysis may include one or more of a number of approaches,such as, for example, knowledge-based techniques, statistical methods,and hybrid approaches.

Knowledge-based techniques can classify text by affect categories basedon the presence of unambiguous affect words such as happy, sad, afraid,and bored. A knowledge base may be used to list obvious affect words(e.g., happy), but may also be used to assign arbitrary words a probable“affinity” to particular emotions.

Statistical methods leverage may utilize machine learning such as latentsemantic analysis, support vector machines, “bag of words” and SemanticOrientation. To mine a particular opinion in context and get the featurewhich has been opinionated, the grammatical relationships between wordsmay be employed. Grammatical dependency relations may be obtained bydeep parsing of the text.

Hybrid approaches leverage on both machine learning and elements fromknowledge representation such as ontologies and semantic networks.Hybrid approaches may detect semantics that are expressed in arelatively subtle manner (e.g., through the analysis of concepts that donot explicitly convey relevant information, but which are implicitlylinked to other concepts that do so).

The state of the art for sentiment analysis includes concepts that canbe used to define what a satisfaction index can really be. Specifically,this index can be multi-dimensional and include finer definitions ofsatisfaction that go beyond the simple “good/not-good” or “like/dislike”dichotomies.

The above-described techniques are usable for text and languageanalysis, but are similarly applicable to image and/or video analysis(e.g., of facial expressions). For example, visual sentiment ontologiesfor emotional mapping that can be used to establish the satisfactionindex. Visual content such as images and videos may include objects,locations and actions but may also include cues about affect, emotionand sentiment. Cues about affect, emotion and sentiment are useful tounderstand visual content beyond semantic concept presence thus makingit more explainable to the user. As an example, emotional mapping may beemployed to identify and evaluate emotions expressed in a digital image.

As an example, the photographer may purposefully express his/hersatisfaction or dissatisfaction by adopting a specific face expressionand this content may be captured by the front camera (e.g., the digitalcamera 102) and the microphone (e.g., the second microphone 104) of thesmart device (e.g., the smart device 100) and later analyzed. Forexample, in order to submit a negative appreciation of a restaurant in asmart device application or through a website, a user of a Smartphonewhile voicing a negative comment such as “service was not satisfactory.”The negative comment would then be submitted to the website as of therestaurant that can be determined from the geographic location of thedevice at the time of the action.

According to an exemplary embodiment of the present invention, thesatisfaction index may be associated with the first digital image 301 asmetadata of the first digital image.

FIG. 5 illustrates an example of a computer system capable ofimplementing the methods according to exemplary embodiments of thepresent invention. The system and method of the present disclosure maybe implemented in the form of a software application running on acomputer system, for example, a mainframe, personal computer (PC),handheld computer, server, etc. The software application may be storedon a recording media locally accessible by the computer system andaccessible via a hard wired or wireless connection to a network, forexample, a local area network, or the Internet.

The computer system referred to generally as system 500 may include, forexample, a central processing unit (CPU) 501, random access memory (RAM)504, a printer interface 510, a display unit 511, a local area network(LAN) data transmission controller 505, a LAN interface 506, a networkcontroller 503, an internal bus 502, and one or more input devices 509,for example, a keyboard, mouse etc. As shown, the system 500 may beconnected to a data storage device, for example, a hard disk, 508 via alink 507.

According to an exemplary embodiment of the present invention, acomputer program product 512 includes a computer readable storage mediumhaving program instructions embodied therewith. The program instructionsare executable by a processor (e.g., processor 501) to cause theprocessor to capture a first digital image 301 of a target object 302using a first digital camera 101 of a smart device 100. The programinstructions are executable by a processor to cause the processor tocapture a second digital image 403 of a user 110 of the smart device 100using a second digital camera 102 of the smart device 100. The seconddigital image 403 includes an image of the user's facial expression(see, e.g., FIGS. 3 and 4, described in more detail above). The programinstructions are executable by a processor to cause the processor togenerate a satisfaction index for the first digital image 301 byanalyzing the user's sentiment related to the first digital image 301.Analyzing the user's sentiment related to the first digital 301 imageincludes evaluating the user's facial expression. The programinstructions are executable by a processor to cause the processor toassociate the satisfaction index with the first digital image 301.

The descriptions of the various exemplary embodiments of the presentinvention have been presented for purposes of illustration, but are notintended to be exhaustive or limited to the exemplary embodimentsdisclosed. Many modifications and variations will be apparent to thoseof ordinary skill in the art without departing from the scope and spiritof the described exemplary embodiments. The terminology used herein waschosen to best explain the principles of the exemplary embodiments, orto enable others of ordinary skill in the art to understand exemplaryembodiments described herein.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire. Computer readable program instructions described hereincan be downloaded to respective computing/processing devices from acomputer readable storage medium or to an external computer or externalstorage device via a network, for example, the Internet, a local areanetwork, a wide area network and/or a wireless network. The network maycomprise copper transmission cables, optical transmission fibers,wireless transmission, routers, firewalls, switches, gateway computersand/or edge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowcharts and/or block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousexemplary embodiments of the invention. In this regard, each block inthe flowchart or block diagrams may represent a module, segment, orportion of instructions, which comprises one or more executableinstructions for implementing the specified logical function(s). Inalternative implementations, the functions noted in the block may occurout of the order noted in the figures. For example, two blocks shown insuccession may, in fact, be executed substantially concurrently, or theblocks may sometimes be executed in the reverse order, depending uponthe functionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

Exemplary Image Capture Scenarios

An exemplary photo shooting scenario may include the following. Firstthe photographer aims at the target either by putting his/her eye in theviewfinder or looking at the front screen of the device (e.g. a smartdevice described above in more detail). The photographer then shoots thephoto. Typically, the photographer then immediately looks at the resultin the front screen of the device, assesses the adequacy or quality ofthe resulting image, and takes action (e.g. move on, or re-shoot thephoto).

In both scenarios below, photographs and voice of the photographer arecaptured through the front camera of the smart device and its microphonejust before and/or after the photographer shoots the photos. Theresulting analysis can be stored with the photo, together with the extraphotos and sounds captured, as described herein.

Exemplary Scenario 1:

Photographs or video images are captured for the few seconds that followthe photo shooting, once the resulting photo is shown in the screen ofthe device. Typically, this would result in a series of images or avideo of the photographer and of the photographer's comments at theirown photo. Analyzing this material in turn leads to an understanding ofwhat the photographer thinks of the photo. For example, attitudes faceexpressions, and oral comments can be derived, and classified asbelonging to categories such as, for example, dislike, satisfaction,worry, or joy. This information can in turn be used to calculate the“satisfaction index” and/or emotion classification for the photo thatwill be stored with the photo for subsequent queries. This exemplaryscenario could also capture the surrounding sound of acceptance or notif many persons are around in the photo.

One possible application consists in associating this information to abroader context—which can be derived from the classifications describedin more detail above—so as to rank the photo among others. Thephotographer may want to first look at photos that have the highestsatisfaction assessment from the photographer (e.g., especially if theviewer and the photographer happen to be the same person). For example,a person may ask: “what were the photos among the many that I shot at agiven family reunion that I was the most satisfied with?”

Exemplary Scenario 2:

In an exemplary photo shooting scenario it is possible to automaticallyrecognize, analyze and register the interplay among people in aparticular moment of photo shooting, in which the photographer wants tocreate empathy or tease a certain reaction from the persons in thephoto.

The following method can then be applied:

First, when the photographer decides to press the shooter button, aphoto of his face from the front camera is automatically taken and hisvoice is recorded, a relatively short time (e.g. two seconds) before theback camera is actually activated to take the photo. This image andsound are analyzed for their emotional content and/or identification ofspecial content, as for example making a silly face to a child orsmiling while shouting “cheese” to a group.

Second, the result of this analysis is then compared to the analysis ofthe content of the photo itself once it has been taken. For example, didthe whole group actually smile, or did the child laughed at the sillyface? The result of the comparison serves as a good assessment of thequality of interaction photographer/subjects.

Third, as described in more detail above, the reaction of thephotographer is captured through another photo and a voice recording,which are analyzed to evaluate if a photo capturing goal was reached.

Fourth, in case of a successful interaction the storage of the momentsbefore, during and after a photograph is captured and a condensed videoof a few seconds with additional information attached as potential querytags is generated. This creates the potential to aggregate all thesemoments: the photographer before, the photo itself and the photographerevaluating the resulting photo.

While the present invention has been particularly shown and describedwith reference to exemplary embodiments thereof, it will be understoodby those of ordinary skill in the art that various changes in form anddetail may be made therein without departing from the spirit and scopeof the present invention as defined by the following claims.

The invention claimed is:
 1. A computer-implemented method of evaluatinga quality level of a first digital image of a target object, comprising:generating the first digital image of the target object using a firstdigital camera of a smart device; generating a second digital image of auser of the smart device using a second digital camera of the smartdevice, wherein the second digital image is generated after thegeneration of the first digital image and includes an image of theuser's facial expression when viewing the generated first digital image;generating a third digital image of the user using the second digitalcamera of the smart device, wherein the third digital image is generatedprior to generating the first digital image; generating an assessment ofan interaction between the user and the target object by analyzing thefirst digital image, the second digital image and the third digitalimage; generating a quality index for the first digital image byanalyzing the user's sentiment related to the first digital image,wherein analyzing the user's sentiment related to the first digitalimage includes evaluating the user's facial expression in the seconddigital image; and associating the quality index with the first digitalimage to provide a quality level for the first digital image.
 2. Thecomputer-implemented method of claim 1, further comprising capturing anaudio recording of the user, wherein capturing the audio recording ofthe user comprises recording the user's voice, and wherein the user'ssentiment related to the first digital image is further analyzed usingthe audio recording of the user.
 3. The computer-implemented method ofclaim 1, wherein the quality index is associated with the first digitalimage as metadata of the first digital image.
 4. Thecomputer-implemented method of claim 1, wherein the user's facialexpression is captured as a video recorded for a first predeterminedtime period before capturing the first digital image and for a secondpredetermined time period after capturing the first digital image. 5.The computer-implemented method of claim 1, further comprising:receiving a query for identifying at least one first digital image of aplurality of first digital images, wherein the query includes a requestto retrieve an image based on a corresponding quality index of the atleast one first digital image of the plurality of first digital images;and identifying at least one first digital image of the plurality offirst digital images based on the corresponding quality index associatedwith the at least one first digital image of the plurality of firstdigital images.
 6. A method of evaluating a quality level of a firstdigital image of a target object, comprising: generating the firstdigital image of the target object using a first digital camera of asmart device; generating a second digital image of a user of the smartdevice using a second digital camera of the smart device, wherein thesecond digital image is generated after the generation of the firstdigital image and includes an image of the user's facial expression whenviewing the generated first digital image; generating a third digitalimage of the user using the second digital camera of the smart device,wherein the third digital image is generated prior to generating thefirst digital image; determining if there was an interaction between theuser and the target object by analyzing the first digital image, thesecond digital image and the third digital image; generating a qualityindex for the first digital image by analyzing the user's sentimentrelated to the first digital image, wherein analyzing the user'ssentiment related to the first digital image includes evaluating theuser's facial expression in the second digital image; and associatingthe quality index with the first digital image to provide a qualitylevel for the first digital image.
 7. The method of claim 6, furthercomprising capturing an audio recording of the user, wherein capturingthe audio recording of the user comprises recording the user's voice,and wherein the user's sentiment related to the first digital image isfurther analyzed using the audio recording of the user.
 8. The method ofclaim 6, wherein the user's facial expression is captured as a videorecorded for a first predetermined time period before capturing thefirst digital image and for a second predetermined time period aftercapturing the first digital image.
 9. The method of claim 8, furthercomprising: receiving a query for identifying at least one first digitalimage of a plurality of first digital images, wherein the query includesa request to retrieve an image based on a corresponding quality index ofthe at least one first digital image of the plurality of first digitalimages, and wherein the query includes a request to retrieve the imagebased on an identified interaction between the user and the targetobject; and identifying at least one first digital image of theplurality of first digital images based on the corresponding qualityindex associated with the at least one first digital image of theplurality of first digital images, and based on the identifiedinteraction between the user and the target object.
 10. The method ofclaim 9, further comprising displaying the identified at least one firstdigital image to the user.
 11. A non-transitory computer program productcomprising a computer readable storage medium having programinstructions embodied therewith, the program instructions executable bya processor to cause the processor to: generate a first digital image ofa target object using a first digital camera of a smart device; generatea second digital image of a user of the smart device using a seconddigital camera of the smart device, wherein the second digital image isgenerated after the generation of the first digital image and includesan image of the user's facial expression when viewing the generatedfirst digital image; generate a third digital image of the user usingthe second digital camera of the smart device, wherein the third digitalimage is generated prior to generating the first digital image; generatean assessment of an interaction between the user and the target objectby analyzing the first digital image, the second digital image and thethird digital image; generate a quality index for the first digitalimage by analyzing the user's sentiment related to the first digitalimage, wherein analyzing the user's sentiment related to the firstdigital image includes evaluating the user's facial expression in thesecond digital image; and associate the quality index with the firstdigital image.
 12. The computer program product of claim 11, furthercomprising causing the processor to capture an audio recording of theuser, wherein capturing the audio recording of the user comprisesrecording the user's voice, and wherein the user's sentiment related tothe first digital image is further analyzed using the audio recording ofthe user.
 13. The computer program product of claim 11, wherein thequality index is associated with the first digital image as metadata ofthe first digital image.
 14. The computer program product of claim 11,wherein the user's facial expression is captured as a video recorded fora first predetermined time period before capturing the first digitalimage and for a second predetermined time period after capturing thefirst digital image.
 15. The computer-implemented method of claim 11,further comprising causing the processor to: receive a query foridentifying at least one first digital image of a plurality of firstdigital images, wherein the query includes a request to retrieve animage based on a corresponding quality index of the at least one firstdigital image of the plurality of first digital images; and identify atleast one first digital image of the plurality of first digital imagesbased on the corresponding quality index associated with the at leastone first digital image of the plurality of first digital images.